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•NOTES FROM THE EDITORS 

— — — ■ • — . — : ■ \ 

This issue of INVESTIGATIONS IN SCIENCE EDUCATION contains a cluster 

S • ' * . ; ■ - } • 

of articles focused on teacher education and a single study. The single; 

" < i • 

study relates to a topic that should be of interest to teachers; the styles 
of questions used in high school biology textbooks". 

Within" the teacher education cluster are. found .ge 1 

written by the same author (s)-. The first three articles appear to 

•'■•■./ * ■ ■ • * ■ * 

'- . * > i •• ... 

redulted^from a single research "effort involving different subpopulatiol 
of the science education community, "too doubt in order to meet space limita- 

- ...... ; fi b . . 

tions imposed fey various professional journals, ttje investigators have 
reported their findings in different articles , ( /e^ch concentrating on data 
related to a subpopulation. Two of the reports appeared in the same 

■ ' ' ' . -\ • • v . \ , . 9 

issue of a journal, Science Education , and have been analyzed by one > ^ 

. " '. • - . . - • ' V*."" \ • ' S 

reviewer. . The third article, appearing ^n Journal of Research in Science 
Teaching , has been analyzed by a different reviewe r '. The two reviewers- 
have slightly different perspectives concerning the articles but thdy .also* 




share common concerns. ^ ■* 

In addition to the first three articles about necessary teaching skills, 
there is a related article. It is related in that , it focused on competencies 
(for "competencies," subs titut^"teachirig skills")^ The respondents involved 
in this research stu^hr were . drawn from within a sihgle^tate/ra£her than 
from membetship^ in professional groups that cut across^state lines. 

There/ is anether cluster-within- a-cluster in the next several articles. , 
These share d common concern: inquiry teaching as well as ' invest'igator^authors 
(Lee, .Lazarowitz) . Two of the articles, on teachers 1 attitudes and inquiry, 

9 ■ 

• : . . '. . ' <j . " . " • 

-. . • • > ^ • 



y 



have been treated tn a single review. One of these articles relates* prima' ^ 
ti£ instrument development ayid the other , to reportirig „on attitude change M ^ 
^measured with the instrument which was developed* Third article also 
* focuses on inquiry .teaching strategies but the educational group involvei 
in thfe study was that of .preseprice elementary education students rather 
than inservice gfecondary school, science teachfers. ^as reported ir. zhe at 



change article), 

-The /two remaining articles in the teacher education jdus£er elate \ 
teacher behaviors. One of these, by Gotten ^t ^al . , investigatea rrje use 



an inquiry/process skills package. The other, by Tamir and Zoor, reprrf 
on the formative evaluation Of a large-scale implementation of an innov^lve 
curriculum prefect. ' «■ ^ ^ 
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Editor 
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Chiappetta, Eugene L. an. Alfred T. Collette. "Secondary Sclera 
Teacher 5kiJ.ls Identified by Science Supervisors. " Science 
Educatium , \2(1) ; o * 71, 1978. 

Descriptors :ucational Research; *Performance Bas^. 

Teacher Educat: Science Edutar-Dn ^Science Teacher^ 
Se :maary TJduci~ .; *Secondary mcol Science; *Skill< 
S. -*eys;- Teach-* - mcation . . 



Expao**d Ar^iract and Ana_"si °repared Esrecially_f or I.S.E. by 
Geral H. — ckover, Purci^e Un^ersity, 

r 

' Purr-- ^ * - i 

^The _-rpoj^ cf Chiappetta a:V: Collette's study was' "to use the p<= ::ep- 
tio~ of «ience supervisor determine those competencies that all 
s£c — mar ^khool- science tf rs should possess 11 (p. 67). Once tjjiese 
coTJarr cen_ ? were identif it .e^ couid "hen be used as part of e-iher 
a r~~--er ^ or lnservice \. 



ser i:- or inservice Vrogram fior secondary science teachers 



/ 



/ 

The authors* have attemptec cd identify initial competencies for use in 

teacher edue^trion programs for secondary science tlachers. They used 

the perceptions of; science sup ervisors for four reasons: 



(1) k Science supervisors have a close associatibn with secondary 
# ^ ^ science-teaching. 

(2) 'Thfcy 'are Instrumental in selecting sci^erice programs anc 
' materials. . ' 

(3) Science supervisors observe science teachers in a variety 

of situations. 

♦ c . * 

(4) They are often consulted in the hiring and firing of 
sfcience teachers (p. 67) . • 

Research Design £nd Procedure 

■ / ' T , ^ . ^ ' \ / 

T)xree stratif ied-samples of 100 members from the National Science 

Supervisors Association ,(N > 600) were selected to participate in the 



reflearchi^ Each sample included a r'aridom sample of 3 state science 
supervisors plus a random sample of 72 other 'member* , ^ost of the 
state supervisors in tlie 50 states were invite'd to rake ?ai^t in the 
project. Ea£h sample participated in only one dfc the :hre£ parts 



t. Ea£h s 

of the 'study. TV * 4 ' i 

Tfie procedures in. this ^tudy were borrowed in ysxt fr-m the^'^De'lphi 
method."^ They deviated from the "original Delphi" ry providing feec 
beck 'to different samples of the same ,pop\^ation, ra/her than t ^ 1 ^: 
sanle individuals. The modifications were utilize^ tn increase 
gpneralizability of the ^ndings. to the ' target popi larJ.cc, prevent 
♦the losd of respondents oyer rounds, an^J" eliminate t— * possibility 
of 'manipulating consensus (p. 68^, The author^ ~£ : hese modifi- 
cations tp help improve the reliability and valid itv of the result? . 



♦ 



i 



;.The research was accomplished using a ^tfiree^round procedure, In 
round oVe competency statements were generated -b^ par-- cipants . 

-In the'^econd round the participants%selected~ levels ca the Taxoncrr^ 
of g>gnitive Objective's that they feljt each ^omp^ enc statement; 
should specify. tti the thi r ' "round the participant /rMorde~ 
tne competencies iti terms : t ...u^orcar: e (p. 68.) . y 




After Tound one, a panel of; five fudges grouped the responses into 
* thri^fe categ/ories 1) cognitive competencies, 2) affecrive compe- 
tencies,, antf 3) pe^jm^lity characteristics. The cognitive eompe- 
tencids^were the focus of the remainder of the" study. After round 
two, one^-sample Xy tests were used to determine the taxonotaic levels 
consistently chosen "by the respondents to^ represent ^the skills. In 
round three the science supervisors ranked the cognitive competencie 
itt order of their perceived importance. - , 

Rank order was determined by the sum|. of ranks^assigned to each 
coi&pfetency.^ !fh^JCenda3?l^ cpef f icient of concordance (w) was computed 
to determine jthe degree of consistency of th/e ranking of competencies 

, :\-; • . . . « ' • • • \ 



Aflfer round cae. - _4 , competencies (skills , knowledge, and attitudes) 
were received ~m 45 science supervisors. * Then a panel „of £ive 
Judges (two 5cier.:e education professors, a science supervisor,^ 
supervisor o~- secondary sciefice teasers, ^rid a, science teapher) 
grouped t the responses into the three categories previously mentioned. 
Eighteen cognitive competencies (34 competency statements) ^ere the 
focus f ot~ thtf remainder of the stucty. \ - \ ' - ~" 

* • • . ^ • ^ ■:• " ' • . •. 

J ^ '• 

In round tv<? . 57 science supervisors responded and identified a 
'level on the Ta^onomx of ' Cognitive Ob j derives at which secondary 

science tea her.3 should a,ble to demonstrate a given skill (compe-v 
- tency* staterrsr.t) , The frequency of^response^/for each of . the ^34 f 
competency statements across 'six taxonomic levMs were %rialyzed 
usftg <&ne-s»mple tests. Thirty-one competency statements showed m 
significance (i5<0.05) in the analyses, while three did not. Thus,"- 
three c et- statements were eliminated and the number of cogni-? 
:ive compr • __es was reduced, fro 



The round^three supervisors ranked the. 15 * cognitive competencies 
fn order jb)f importance. • Rank order was determined by the t sums of 
ranJcS^as signed to each cpmpetency. «' Tfre Kendall coefficient of 
concordance (w) was 0.29 S (p< 0.001) foS« the ranking. 

. ' .', • & . * - -.V 

The first eight ranked competencies appear .Cfe be important* and 

necessary foir'science teachers.. "N^ey are: . - 



) formulates instruction that ^emphasizes the" inquiry "and. 
process approach to teaching 'science, . • ♦ 

2) plans and organizes appropriate instruction, « '- 

3) ^provides <*for individual differences, * 

4) etnpl<j(ys a varietyitof instructional strategies au4 
techniques, '* L* \ 9 ' 

5) evaluates student prbgress arid success, * ^ J' 

6) demonstrates effective management 05 the -science 
^laboratory, ' - / * 



^ 7) v interprets the psychology of 3fearjf|pg as it ^relates' to 

science instruction, - ' . j . \ "~T" _ 

* 8) organizes ,a^,relevant science cjurricul^im. - ^ 

: v' ^ . .. • 



Intelqftretations ; " > * f . * * 



.The -eight conrojpte'hciers^ prieviously stated appear to be important aijd" 
necessary fop s^i^^e teacher^^-These competencies represent f a core 
'q| essentials skills that should be the foc'us of inservice and-^re- 
service^ scfiehce teacheVj training, Th&\authors also "state th^t it^ 
would be Significant if the professi^fh could facilitate ( the achieve- 



ment of; the* eight competencies by a majority |jof secondary science 

. /* - ■ ^ 

teacher p. . • 



^ ' : ABSTRACTOR'S. WlYSIS ^- 

CKlapp^jtta and Collette's study prpvfLdes^ an excellent foundation for \ ^ 

* future" research and discussion with regard to jthe £d«entif icaticm of^A/^ 

, ' teacher attributes related - to effective- teaching. Ho'Wever, #rie ques- 

• * ■ V ' ' v I ij - • ' s 

, tiotis whether science supervisors are the most competent to -identify , 

■* ^ • i * ' 

. the' skills needed by the secondary science teachir,^ furthermore, * *y 
1 .' \ y * r 

is a sampling of member^ of the Nationalise i^nce Supervisors Asso- 
ciation a random sample? The response rat^ oi; usabl«*^returns from 

' \ . ' D 1 ■ ' « ' ' * * , C &s - ' 

fcljfe science supervisors for the three roujids was only Jk5 percent, 

57 percent ,, and.55 p^rcarj: respectively with over 90^ perceht^ of the > 
^ respondents t^ing t^ales/^ Also one tan irtfe^r front -the jda£a presented 
that over 50 percentNjf^he jes^ondefcta had not, tgught science, af t^er 

v i97i: • v * "\ S r < ; * : x 

"While^the modified Delphi ipethod was uied to 'improve the validitys 
tad reliability of the results, the resists dp not seem to support ^ 
ttiis. Furthermore it? is disturbing to note that, \Jhile th6 "judges" 
"grouped the responses into three categories 1) cognitive, competencies 
2) affective competencies', and 3) personality characteristic^,.* the *' 
authors chose to ignore two of the three categorizes ahd y only jtdcussrd ' | 

' • • * ' 6 ■ - N - 



i 

f 



- | 




upon 9fagnitive compertepcies, ^hereby negating two-tfrirds of the cate- 
gories a^^ap^roximately 30o2^tatements; Their reas.ons^for ignoring 
affective "competencies and personality cfiaracyerist'i^cs are. glaringly 
'weak add distract froto the us^ailness 'and dredatu^Tof % the §tudv, \ 

;' ^ \ V , • . \ - • '. . : v 

Even -conjc6tU rating in the cognitive competencies and utilizing the,- . 

» \ ^ w,p ' ~ , . . • 

Taxonomy of CognitiVe Objectives, the authors made 'an •additional 
^assurtption," " The author^ assumed thqt ,th# six 'tax&noinic levels 
have an equal" probiki^ity of select %oti (p;^3). Tajconomic .cl^ssif id- 
eation indicates thaft this assumptions^ not valid and that nearly \ 
' 80 frercerit eft the iiAnitive selection \s inr-the bottom three level?. . 

The Kendal^ cat^ic lent of c concordance (w) was 0U29 . (p ^ 0,001) % \ wh^ch 
Suggests a rather lpw, degree of cdnsistencv amorfg .the" racking of, the 
15 competencies (p.' 69).?- Furthermore, the authors* state that the ^ 
flrs^ eight competencies appear /to be important aftd necessary for* 
science teachers (p. 70>. They base file dividing lirfe b f ecaus^ of , 
a "gap" «*f 50 points bet^efen competencies eight ajfuj nine, J .If one"/„ 
examines Che data table, o*th$r "gips" can also be detected such as 
a 27-point "gap" between ^^P^tencies . two and* three^oand ^ llS-jJoifrt 4 
,|f gap. lta5 t>etwe5n competencies four and five, Tflie authors have . failed^ 
jto identify wh£ch> competencies really the^moSt important. Are . 

* there really *ei£ht or J^ust two.oT fifteen' or ~none? »" . ' *, ^ 

^f one examines^he first two compet|iicies which are ^separated by # 
onp point the^ are: 1) formulates instruction that emphHrsrfzes thfe . 
inquiry and .process approach to teaching science, and 2) plans and 
organizes appropriate instruction, bne wonders if. these are truly 
"cognitive" "competencies a^deilineatfed* by thfe Taxonomy^ of Cognitive 
Oblectives. If -these are cognitive competencies^ wh£t did^lTTie" ) 
afftfc^tVe and "personality ^cetfn^tr^ffcies look^like? It ( is;debatable ^ 
that' these, or uo to one-ha^f of the competencies: listed are really 
"cognitive" ones. * * 

, \ . ■ L 

Future studies snouli^focus upon the identification of^secondary 
science teacher \^cills utilizing all three domains of learning 
•(cognitive, affective, and psychomotor) Furthermote, researchers 




Chiappetta, Eugene L v and Alfred T. Cdllette. "Secondary Science 
Teacher Skills Identified by Secondary Science Teachers." 
Sci^ce Education , 62(1)V J3-78,' 1978/- 

)escriptors — ^Educational Research; ^Performance 
Teacher Education; Science Education; ^Science T,eichers; 
Secondary Education,; ^Secondary School Science; *Skills; 
Teacher Education - m . 

r • . . • . * " , ■ • 

Expanded Abstract and Analysis Prepared Especially for I,S.E, by 
Gerald H. Krockover , 'Purdue University, '. .■ . *' *. 



Purpose - • * , * h 

\ \ * 1 • ' i 

The fl purpose of Chiappetta arid Collet te's-- study -was "to solicit the * 

perceptions of secondary School science teachers to determine those 

• • s < 

competencies that all secondary school, science teathers should 
possess" (p. 73). It was hoped that,' once the competencies were 
identified, they could* fc^e used as part of the secondary Science 
teacher education program. " . > 



Rationale , * ■ 

The kuthors have* attempted to identify the needed science teacher 
skills as perdeived by practicing secomdary science teachers v The 
authors recognize that ciifferefrt: groups Cscience supervisors, science 
educators, science teachers, science, students, etc.) pirace different 
priorities on the desirable attributes of science teachers, 

• .. . - r * ' • • . 

■ > . - * ' - . * 

' * . \ ' ' ' ■ r 

Research Design and Procedure ' 

Three random samples of 100 secondary science tea^ers^from New York 
St^te who teach in grades 7-12 were invited to participate in the 
stuciy. Each sample was involved in only one of the three parts of 
the study, " '■ 



The procedures utilized a modified Delphi method and the * research was 
accomplished using: a . three-round procedure. In the first rpundl compe 
tency statements were generated by participants. In the second round 
the participants . selected levels on ttfe. Taxonomy, of Cognitive Obgjec- 

** v ■ ' ^ ' ' ' 

tiveivat which they felt each competency ^statement should be specl/*- 
fled. In the third rbuhd, the participants ragk-ofdered the-* . 
coiwfetencies in "terms of importance. / • V, * , r 

. . ■ ■ . ♦/ \. , ■ e * • 

After round one, t;hd judges, (two science education prof essorg., a 
science supervisor, a supervisor of secondary Science teachers, and 
a science teacher). -.grouped the responses into three categories: 1) 
cognitive competencies, 2) affective competencies, apd 3) prersonality 
characteristics. After round two,, one-sample X tests were used to 
determine the taxonomic levels consistently chosen by the respondents 
to represent the skills. In round three, the science teachers ranked 
the cognitive competencies in order of their perceived importances 
The Kendall coefficient of concordance Was computed to determine the 
degree of consistency in the ranking of the competencies, 



Findings / ' 

■ ■ '. . / • •• ■ . ■ " - \ \. . ' 

After round one, 370 competencies were received from 48 secondary 
science teachers. The panel of judges grouped 'the responses' into 
the three categories previously mentioned. Fifteen cognitive compe- 
tencies (34 competency statements) were the focus for the remainder 
of the study. 



In round two, 52 science teachers responded and identified a level 

on the Taxonomy of Cognitive Objectives at which secondary science 

teachers should be able tp demonstrate a given skill (competency 

statement). One-sample X tests were used to determine the taxonomic 

levels consistently chosen by the teachers to represent the, skills. 

2 

Skills or competency statements with assbciated X values significant 
art p< 0.05 were written at the specified levels. Thirty competently 
statements showed significance in the analyses, while four did not. 



In round three, 54 participants! ranked the 15 cognitive competencies 
in order of importahce. The Kendall coefficient of .concordance w^s 
'computed to determijie the degree of consistency in the ranking of 
the .competencies and. a concordance value of 0,26 (p < 0.001) was 
obtained. „ . 

• ' - , : 

Competencies rAnKed numbers one, two, and three appear to be most 
important tp the secondary science teachers. They place human rela- 
tions first, inquiry teaching second, and employing a variety of . 
instructional techniques third. Competencies numbered 4-12 appear 

to occupy a moderate degree of importance for secondary science * 

• . u ' . „ ■■•>' - ... v - 

teachers. The areas related to these competencies are the psychology 
of learning, ' organizing instruction, the laboratory, evaluating ■ 
achievement, science and ^society, a generaI\^ducation, classroom 
communications, and classroom discipline. Those competencies ranked 
/ l3-15 seem to have the least importance. The science teachers appar- 
ently see little need for improving teachers 1 background in the 

sciences. At the "bottom" of the list of priorities is the area 

* * * * ■•**•.." 

of professional growth and professionalism. 



"Interpretations ^ * \ 

Three of 15 competencies, according to . the authors, , appear to be most 
'important" to science teachers. The authors al^o^expressed surprise 
that science .teachers rank human relations first, arid professionalism 

. ■ * . * ' * :* a , 

last. The authors alao .speculate that what is taking place in * 

»■ • • ■ 

secondary science* classrooms is psychologically stressful for students 
» ' . • v . . " , ' \ 

ahcf^teachers. They also state that "science classtooma, in general, 

probably lack an environment which promotes science' a^ a human 
.endeavor" Cp. 76) . 1 They conclude that "science educators will have 
tp improve science teach^es 1 attitudes toward becoming more active 
professionally before secondary science teacher skills <an be signi- 
ficantly -influenced." 



ABSTRACTOR S ANALYSIS * 



Many of the criticisms cxrected at Chiappetta .and toilette's science 

K c ' 

supervisor study also hold for thifc study. This study asks more 
questions than it answers and the authors seep .Surprised by the 
results. The response rate of returns from the sample ftfr the 
three rounds is similar to that for the supervisors study (48 percent, 
50 percent, and 54 percent respectively). Also oyer 80 'percent of • the 
respondents* were male and all were from New^fork State, Is New York 
a typical sta^e? The authors define secondary science teaching as v 
grades 7-12 while jpany other states define secondanjr science teaching 
aq grades 9*12 'arid junior high/middle, schgol as grades 5-8, fDid the 
authors lose something in trying to cover graded 7-12 rather than, 9-12' 
Were all grades equally represented in the "sample? t The authors failed 
,to report this. ^ - 1 * •« , 

The validity and reliability of the modified Delphi method is just as . 
questionable as in the Supervisor study along with the authors' use of 
categories arid limitation to cognitive competencies. In this study" 
the authors also assumed that the six taxonomical levels have an equal 
probability of splection. The Kendall coefficient of concordance (w) 
was, 0.26 (p 5 0. 001) , which is even lower than the value reported in- the 
science supervisor study. 

The grouping of competencies is also as questionable as in the super- 
visor study. There. is a "gap 11 of 46 oirits between the two "top" 
competencies and yet the authors group them together. However there 
is "only a 51-point gap between competencies 2 and 3." Again, *tfte 
authors -have failed to identify which competencies are really the 
most £mportaiit. 



\ 

As in their previous study, the authors call these competencies 
"cognitive." How can "providing for a humanly supportive learning 
environment" (competency 1) be ila^sed as cognitive? Are the compe- 
tencies identified by either or these studies really cognitive? 



r 



\>ere surprfrs 



The authors surprised that the f^condary science teacher ranks 

hukan relation^ a solid f ^*st . as opposed *«to science supervisors who 
rank inquiry and process ^:>proach* to teaching science as number one 
This indicates the values held by each group. . * 

Ev^rf^though these two studies have raised more ques tio^s than they 
have answered, they are Valuable additions* to the science education 
literature reservoir, "ijhey should serve to cohtiitoe to promote and 

/ r - / ■ • 

provoke science education persons from all^facets of educatiQtial 
levels to continue to try to answer several questions; "What should 
be taught to the pr ^service and in^ervice scifericd teacherjg" "What 
coqjpetencies are .of Xvalue^ tp the secondary science teacher and can 
they be ident^ied?"^A^dii:lLonal support should* be given for furthei 
studfes that will attempt to delineate those 'skills that are viewed 
as basic competencies for the secondary science teacher, tfe have 
not yet developed an adequate ranking of skills or competencies 
needed by secondary science- teachers, x>ut it's a beginning. 
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Purpose . < • ' ^ * A 

r ■ I "... . - . * . ' 

•The primary purpbse of the £tudy was to improve ^upon previous attfcagpte 
to ^^ntify specific skills necessary for teaching science ifi the; 
Secondary school. The investigators note the lack qf^esearch which 
clearly ident^ififes skills essential for science teachers — at least 
skills where there is general agreemeht among science educators. They 
set out to establish s^u^h a listing by exploring a reliable method 
and a valid population for identification of such a set of competent 
cies. The investigators suggest that; such a set of cipm^tencies or 
skills will provide "direction for experimental research and for 
science teacher training." Hence it can be stated tha^f a purpose of 
the investigation was the establishment ^f^a researclfba^e which might 
influence collegiate urograms designed to prepare science teachers. 




Rationale 

f, 

Specification of knowledge and activities "necessary to teaching" is »' 
purported to be central to teacher training. The authors report 
that statements identifying- goals, objectives, competencies, an<f 
skills have been developed by teacher educators. Few question the 
value of such statement's; however, there is often controversy concern- 
ing their validity . The authors state that the controversy arises 
concerning the studies from which such statements of goals, objec- 
tives, cpcipetenci^, or skills are concluded. 

14 ' ■ • ** ••' • ' 



Several references are cited (Inc Gage, Rosenshine, Balzer, < 

Evans, and Blosser) which suggest a l£ck of direction to teacher edu- ' 
cators ^for their preparatory programs*? the lack of ^sys.tematic and 
general iza£le studies, and . A the poor quality of many of'the research^. 
' reports concerned- with science teacher education, * s 

Research Design and Prqcedures \ ^ J - 

Nearly half* of the membership of the National Association* for 
Research in>-Science Jeaching t (NARST) was r^ndojnly selected. One- j 
third (or 1.00) of th^ sample was randomly assigned to each of fihree . 

♦groups for involvement in one of -three' phases of tbe study. 

> - • •' ' s> > 

A motfif led Delphi ^technique was**iised for analyzing the^iata; different 
samples of the same population x ( three groups of 100 NARST members) pro 
vided feedbackfcseparately at different times and stages in the study. 
Such a modification purportedly increased the generalizability of the 
/ results to 'science educators, prevented tHe loss of respondents over 
time, and delimitated the possibility of irianipulat ing consensus,. 



In the first round of the^investigation competency statements were 
generated by the first saifcSke of NARST members; in the second round 
the second sample of NARST Vmembers was asked to select levels at which 
teachers should be able to demonstrate a given skill; in the third 
round a third sample of NARSTynembers. was asked to rank-order the 
competencies in terms of importance. The perceptions of the three 
samples of NARST members were considered the independent variables 
with the products generated at each round tliej^J^pendent variables. 

During round one the randomly selected sample was asked to identify 
competencies (skills , knowledge, and attitudes) that all scifence 
teachers at thfe secondary school level should possess. Sixty-one 
pfersons responded, with 432 s tatements ." Five .judges we'reT selected 

sorting the responses into common categories; cognitive, affec- 
tive, and personality were the categories selected. Thirty-nine of 
"the statements were assigned to the cognitive category and l-ater 



/*.(--• 

gromaed into. 17 cognitive competencies.^ Eachl^i; the^se 17 .titles was' 



use 



t| * *ix 



org^ni^e tfievcompetency statements -or • indicators **Mch* speci-' 
fieu jcnovledgp ancl skilly perceived as jafeces^ary for sVie:^ v ^chiiig. 
These cognitive competencies were \is&& as the focus for t)£ * *ire^ 
stu<$ since "they can be develape/d through .teacher '"train 
aince they comprised the ,? buik of ilsable~ resppnses . " ^ 

, ' .2 ■ ■ ' .'• ■ V* ' 

During round two the second randomly selected sample was asked 4 to 

" 1 ,*• 

identify ^ level ^is^ng Bl6om ; s ^Tfcxonomy of Cognitive Objectives for. . 
fractv* competency -statement. The ^^tements were "stripped o-f^ terms " 
which voul-A^!ink themato one of the ^axoncimic levels , namelv know- 
^letfge* compretiension, application, analysis, synthesis, and evalua-* 
tion. A total of. 45 of the 100 persbns sampled responded. Statements 
which were^nbt consistently assigned a single taxonomic level were 
'if"' **ed from the stud- because such statements would be subject to 
*vj*. nterpretation by the total audience. After round two, 15 

c jnitive competencies r^ma^^ned? 

During round three the third randomly selectee sample was provide^/ 
the list of competencies which remained. ^EacH^consisted of a title J 
and Que or more competency statements . m Fifty-two persons responded^ 
and ranked, the competencies in or^ier of their perceived importance. 



Findings < 

The major finding cz tr study was the ranking of the 15 competen- 
cies, after rour_- three f the^S tudy . The rank order was .determined 
by the sums of rhe ranics assigned to each competency. Two separa- 
tions in ^ie ranking occurred . \rhe third and fourth competencies 
are separated t" 61 joints,, and 1.2 and 13 are separated by 78ppoints, 
Hence it can be stated that the first three competencies' are defi- 
lit^y the most important with competencies 4 through 12 the next 
important -^competencies 13 though 15«are least import ant • 



(/ ' \ ? • " - 

s ohe* rank order of the cotapetencies is*,as follows: 

1) Pro^»tdes 4 for a, humanly supportive* environment: v ^ 
v ?) Plans and orgai^zes instruction; ^ V ^ \ ^ 

* ^^ Sss 3)" ?os^essesl*a sound science subject matter background 

^ 4) Relates ps^chol^oglcal % development wi'th the* lea'tning^of «» * ^ 

* 3cienc^ subject ^natter; * ^ ^* 

5) . Demonstrates t^e ability^ tQ Communicate effectively; 

6) , Incorporates effective latoratory^ activities* intp' * 
' e instruction^ V ^J* ' , " 

*^7) ^Teaches science j^ing Ae inguir^, pxo cess, and ..discovery 
"xapproathes:" ' k N , 

; \8) Uses a variety of # instructional^trategies and tQchrWques; 
90 Provides for individual differences ; , 1 

•O: 

10) Employs evaluation skills ana procedures ; 

11) Employs effective classroom management techniques; * 

12) Relates science to society; 

13) Possesses a knowledge of the history and philosophy of 
science and of its sbcial implications; 

14) Possesses a back^roui^d .in mathematics;) 

15) Knowledgeable aDOut ^^rl^u^a^ience curricula. 

- - „ , a 

The Kendall coefficient of co ^ordance was computed to deT^rmine tha 
degree of consistency *b the ^ranking of the 15 rc^mpetencies . A 
rather low concordance, namely 0.23 (p^O.001) was')found. This 
suggests a low degree c consistency in the ranking by the third 
sample, of respondents. \ 



Interpretations 

Since the 15 competencies are similar to those reportec using a con- 

sensus model, the authors repajrc that "conventional wisdom 11 has 

identified many skills that/need "validation in the classroom. 1 ' ^\ 

* * "*•-■■/■/ 
The ^uthorsj call for research thaL wdll relate teacher competencies 

to student achievement and \stucient attitude toward sciences They 




call for the development o ( f research instruments which will permit^ 
researchers to measure the degree which spience teachers can demon- 
strate the skills.) 



I 



,The. investigators report that ~ of th *^ 5 cognitive competencies^ 

can%e dfeWlo^Ted in in-service o Acraduat^>rog^ains Further* they 
urge that ^sev^rad af the competencies should become th? focu^ for 
^secondary science ; teacher ^ucationy^ Science teachers uomp^t^tlt in 
► tjie>lf> a reas^ll 1'' encourage experimental research f on teacher efjfec-^ 



^ : ■ - /Abstractor's analysis • * 

This study represents a new dimension, in assesjsiag n^for pompe^eacips 

• 4 " •' v . 

with input from a significant p^rce||&ge of- a m^ Jor research a$sc«51^j 

tipn in science education.. re identification and ranking ofjl5 

necessary competencies for teachers of seccmdary "science /i^s^worth- 

while contribution. The results can be compared with o£her reports — 



mos^ resutt^iQjg from creative endeavors, panels of -experts, or other 
less abjective m£ans. The fact that the results of the =;tudy compart 
favorably with other attempts a: identifying needed comr cencies for 
science teachers, is reassuring. It also means that the ^tudy has 
not resulted in any really new conceptual coritr ibuticns for science 

education/' . \ \ 

, i 

-r 

The use of a modified Delphi techniqu: is of Interest and a valuable 
methodological contribution, i'^ing a significant number >f science 
educators who are members of a research society such _ lMARST is 
desirable and affords some uni:ue possi'r ilit ies for design and analy- 
sis. Some .opportunities for correcting concerns for reliability and 
validity in such studies of perceptions were not Examined, however. 

r\ ' ' 

Several questions arisfc from the manuscript, the design, the remits, 
the collusions* and' recommendat ions . Certainly there can be some 
questions' raised concerning the assertion that NARST members are pri- 
marily concerned with tocher education and that significant numbers 
conduct research on t^gher preparation. Their depth of understand- 
ing of ^teachers and instruction could bfe questioned as wej.1. When 
only slightly over halfof 300 randomly selected members chose to 
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rfspc&'d,J other questions c6ncernin£\tfieir interest -.in teacher education 
^depth" of Understanding, and ^unique" suitability for this .kind of study' 
arise. . • - • ^ a • "l ^ i 

' . ' - 'V... v * ' v 

Tha authors' failed ^o capitalize f ully-onA:he modified Delphi technique 
They did not show.-hav' the use of three separate groups ; for ^ach. phase v 
*jqf the study t£e r^sul^S more generklizable. ^ ♦When nearly 50 per- 

cent, of the sknple was lost: in each phase^/ttte claimed 'farevedtioo of 
loss 6>f respondents dver ytourids" seems like a rqu6stfonable*advantage. 
Wbuld not a gi$pu£ -selected because of (their *specif ic involvement' in'" , 
— ^tradher educ&fcioh have been ^ fetter choiee for j^actaht sample 0 ? jpf »^ 
teacher' education ^rgani^at-fon^ (AETS) f , a^ grotf^associl^e(i-w4^i 4 f unded 
ptbjects fors science teacher education (UPST£P)^pr a\„group identified 



with. promising practice^/ could \perhaps have /provided a better sample 
group for responding,. * , -\ v \ r 



The three tasks asked the three sample groups seem very divergent. 
What is the advantage of -having a groub generate 432 competencies? 
Why another group lor classifying the, cognitive levels? How are] 
the generation activity sandj^he classification Activity related,, to 
the tanking activity? How do we 1aiow r that another sample of NARST. 
members would have generated the same results? And yet the findings 
of thV study r .represent a list of If* competencies ii^ rank order — -by 
a groupl of 52 NARST members. What would the 43 N^ARST members .who » s 
di^ not rank the competencies have done? This seems like^a legiti- 
mate question when ^tee degree of consistency from the 52 .respondents 
is so low. |s it- really fair. to comment, "One mightnot expect a 
higher assbciation to be: found among: 55 sets of rankings for 15. \ 
items 11 ?. Perhaps this la<^ of consistency deserves more attention. 
Where were the differences? Who were trte^eople? How do their views 
of teachers and instruction compare? Since it is assumed NARST y 
members have a d£pth of understanding of essen^ iaf skills^ for 
teachers of "secondary . school sc ience, is^t fiecessarily so tha£~ 
this ''understanding" is the s^me or even Similar? - A 

Id this work "of the five-person pa.nel of judges a desirable part of 
the research design? Would there not be more validity to using 
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anfrtfre^grCup of NARST^members? * ( Why us*£ a^scienCe coordinator, a 



teacher, and a -student teacher supervisoti as a" panel of judges? What^ 
Was the purpose of the tha-ee-cySYegory division tha^the judges prc?-^ 
posed? 'V Since ^il was ighoced except for the ^gnitive^competen^ie^, 
could 'not t£e. Study ">have started. at this point? < Wha£ i^B^nt by the 
sratement thafc the cognitive competencies * "compr ised ' the bulk of the * 
usable,, responses 11 ? Wh^t 'is the,eVidence that ycognltfive competencies 
jepsesentf "" sk H^ s which c'an be developed through teacher training 1 * ^ 
as oppoged^ to other cbmpe^encies? % . 4 . ^ * ' . ■ 

*What*made the authors^feel that the responses in ea$h ^rank were repre- 

h % " ■ • * 

sepffcative of the perceptions of ■ t^ie target population? Could nof^a 

check he made? What 'about other NARST members - ?* What^ about noTi-^ 

respondents? Was other information collected on those choosfng to f 

respond? 



is 

a 



The authors, contend that all competencies, identified, in'* the study 
"should be considered by science educators in their research efforts 
on science 'teacher compet^ce and s'cience teacher effectiveness. 11 % 

What is the' nature of °the other competencies' that were "proposed? f It 
"is fine to state that* ail should be considered but what is the 
rationale? This seems especially strange when the authors admit 
^hat the 15 competencies are not all that different %rom a number 
of other lists th^t have been generated in other i/ays* Perhaps a 
question. concerning the v^lue of such global and general lists could, 
be rteJLsed. How does one do research knowing^ that .52/feRST members 
rated ''provides for a humanly supportive environment 11 as the most 
important : cognitive competency f<jr secondary science teachers? 



After choosing'to classify and to rank cognitive competencies, the 
authors conclude, that affective and personality ..characteristics are \^ 
important variables. .Perhaps this whole dimension should have been **" 
omitted by definition #r it should have been developed ttore care- 
fully. For example, what was thfe nature of some^competencies in ^ 
th^e categories" which were identified in round' one of the stiudy? 
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*To contend that jtjie 15 cognitlVe cpmpet eijgies."6an be developed iir 
extensive in^st^rvice $p graduate programs seems inimateriVl . And 

— tsJia is followed yith a sentence that elg^t of thfe 1*5 couid became 1 
a focus for a pre-service prograjn. Why eight? Why the focus^ at ail? 

. Because 52 KkRST memberS agreed \^'rank ttfe 15 sftateynentsl^ *^ 

\\ • • w ' ' ' \ - "t 

• M ' ' ■ . 7/ * r± . 

The closing senterifce^fic f£rs £ome question. Why will the production * 

of riewj teach^ts' who have the 15 competencies increase the i*robabili£y 

of experimental research on secmQav*? science^teacfier effectiveness? 

. The Rationale needs N to be expancj^a, explained, ©r illustrated.'' ^ 

ftesearch concerning^perceptions, of pifof ^^ibnals regarding teacher 
"Characteristics remains interesting and valuable to a -degree. l<ike 
so much educational res^rch, more precision and more specif icity. is 
needed. .. We need to know the backgrounds, the positions and the 
-philosophies* of a given ^^up of respondents . /Wouldn/ 1 there be a 
vast difference in the perceptions of a respondent who had been 
, involved with humdn relations anemone who had not? Such differences 
could exist ~ and probably do ~ fo'r each of the 15 competency areas 

We need more rationale — a better framework ffi^- from which to ^oaduct 
our' research . Perhaps agreement as to 15 or 25 general competencies 
is fi^e. Hpwjetfer, to be meaningful" aim useful these must Ve better ^ 
defined and exemplified. To affect tea&ier education, ^to influence 
science teaching, we need to do more than to classify statements and 
to rank general competencies as to iinportance. , 
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PuT£Ose . . . J 

The pur$6se of^this study by^ But2ow and Qureshi was to identify and. 
validate observable teaching competencies for use^in -the preparation 
of prospective high school science teachers. * 



Rationale 



In * the past decade, the growing demand by the public for greater 
teacher accountability has^given rise to attempts by educators at 
all levels to identify teacher competencies that are both demonstrable 
and valid. — The investigators have stepped into this milieu in an 
effort .to provide a substantial research-based footing l:or the develop 
ment of teaching competencies appropriate for use in the education of 
secondary school science teachers. Most of the effort to. develop 
competencies is being carried on wit?Rin state departments of education 
teacher education institutions, and individual school districts. 
While this effort is national in scope, it is highly fragmented with 
little agifteement on what competencies to include ajxA what to exclude 
at the various stages of a teacher's development. In an attempt ,to 
go beyond the extent lists of competencies, the' investigators have 
based the development of their proposed competencies^ 09 the statements 
rankings, and observed behavior of science teachers identified as 
highly competent.. * . 



Research Design and Procedure 



The subjects were 21 high school science teachers selected by a strat- 
ified random sampling of science ceachers Identified as being highly 
competent, by. a statewide (Maine) rand&m sample of "201 teachers. 
Stratification was by county with each stratum representing one of 
the 16 counties in Maine. 



Thfe study was conducted in two parts. In part 1, each of , the 21 sub- 
jects was questioned in an interview about their ideas of a "competent 
high school science teacher i 11 Competency, as used in the study, was 
defined for the, stibj'ect by the interviewer. Following the interview, 
each subject was requested to rank order the list of competencies 
he/she had developed during the interview. The audiotape recordings 
of the interviews were reviewed for additional competencies mentioned 
but not listed during the interview. In cases where competencies were 
added to the lists by the/investigators after a review of the audio- . 
tapes, the amended lists were returned to the respective teachers for 
re-ranking of all, items. 
' I 

Finally,; a list of \J. generic competencies was developed by. the inves- 
tigators from ^n analysis of transcripts of each of the 21 interviews. 
T?he subjects in the stuc^ were then asked to rank the 12 generic compe- 
tencles. Kendall's Coefficient of Concordance (w) was used to calcu|.at 
the inter-rkter agreement of the rankings, The w value of 0,40 was 
significant at the 0.01 level with a standard deviation of 48.12, A 

test was used to determine whether there was a significant differ- 
ence between the competency rankings. Compared agains,t each other, 
no" significant differences were found between the rankings of adjacent 
competencies. The competencies are listed below in ordef from highest 
to lowest rank, \ . 

4 * 

1. A teacher must know his/her subject an'd keep striving to 
update his/htr knowledge. 

2. A teacher must have a good rapport with his/her students, 

3. A teacher* must recognize the individual academic abili- 
ties of his/her students and^ry to encourage each of 
these students. s \. ^ 
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4. A teacher must make his/her lessons interesting without 
wasting time. .© s 

' 5. Ajteacher must be able to control discipline problems in 

ojfdet to protect the learning expetlence of his/her .students. 

6. A teacher must plan class lessons in advance with the idea 
to present scientific concepts and ideas in an organized 
and clear mariner. - V 

A teacher must foster unbiased, independent and critical v 
thinking in his/her students. . , ■ 

8. A teacher, especially the one who is teaching low or 
average ability students, "mils t relate the scientific 
ideas he/she is teaching to the, daily life experiences 
and the needs of the students./ v 



9. A teacher must evaluate, in a traditional or any fl| er 

practical sense, the academic progress .of his/heifl^tudents 
and make the results available to them as soon as possible, t% 

~ • - ' • ''i*' ' - 

10. A teacher must, appropriately, respond to the sudden diver- 
sions of students' thoughts. 

11. A teacher must take appropriate actions and instruct 
students about laboratory safety practices. 

12. A teacher must fulfill his/her professional responsibili- 

■r ties. ' • f 



Part 2 of . the study was designed to identify "the extent to which these 
teachers used these competencies..." Each of the 21 subjects was video- 
taped for 10-20 minutes while teaching. Most of the videotapes were 
made on the same day as the interview. Following the videotaping, the 
investigators divided each of the 21 videotapes into ' five-minute 
segments. One five-minute segment was randomly selected from each 
videotape for viewing by judges. A panel of U4 judges was asked to 
view each of the 21 fiver-minute videotape segments and to rate the 
competehcies^sing a four-point scale of 1 (superior), 2 ( satisfactory), 
3 (poor demonstration), and 4 (no demonstration). The panel of judges 
were all members of the New England Region of ■ the Association 'for the 
Education of Teachers in Science (AETS) . All ratings were made inde- 
pendently by the judg The judges' ratings were used to validate 
the competencies and ao a basis for identifying a final list of compe- 
tencies. te 
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Findings 

Inter-rater agreement among the judges wad calculated for each of the 
12 competencies using Kendall's Coefficient of Concordance: w values 
ranged from 0,24 to 0.77 and all were significant at the 0.01 level'. 
Standard deviations ..ranged from 8,94 to 15.82. 

Chi-square was used to test the difference between the number of times 
a competency was demonstrated versus the number. of times not demon- 
strated. Competencies 1 (knowledge . qf subj ect) , . 2 Cg<?od rapport),' 4 
(making lessons interesting), 5 (discipline)., and' 6' (lesson 'planning) 
were all demonstrated at the 0,01 significance level. 

Competencies 3 (individual abilities), 7 (critical thinking;), 8 - + 
{relating ideas), and .12 (professional responsibilities) were non^ 
significant at the 0.05 level. 

Competencies 9 (evaluation), 9 10 (responding to sudden diversions), and 
11 (laboratory safety) were nort demonstrated- at the 0.01 significance 
level. 
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In t erpre ta t ions 



The investigators found that a set of highly rated science teaching 
competencies could be generated by experienced science teachers and 
judged demonstrable during instruction. It is interesting to note 
that fi/ve of the six. highest rated competencies were judged as signi*- 
ficantly demonstrated, whereas, three of^ the four lowest rated compe- 
tencies were judged significantly not demonstrated. The investigators 
concluded that pre-service and in-service training programs "should be 
designed to promote such competencies as primary goals." An extended 
description of each of the five validated competencies was included 
in the conclusions. > 
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ABSTRACTOR'S ANALYSIS - - « 
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ithe investigators are to be commended for their effort to identify an 
important but mapageable generic set of secondary school science, 
teaching competencies. The teacher-generated list reported in this 
study, however, is nfct- unique in that each of the 12 competencies 
indentified may be found in other such lists and is applicable, with 
the possible exception of laboratory safety, to many other areas of 
instruction. One must decide, then, what this research contributes 
'to the growing list of products of the nationwide effort to identify 
for both training arid accountability purposes valid and demonstrable 
teaching competencies. I would agree with the investigators that we 
will probably never . develop a set of competencies appropriate for 
all situations. . 

Since the identification of teaching competencies does not lend itself 
readily to a research design, one must examine With great care the 
investigators 1 method for generating the list oft competencies in order 
to make some^ judgment about their usefulness. , 

Firs"t,* the terms competency and validity occur frequently in the 
research report and ar^ central to the study. ^Competency however, 
iia never /ie fined or thfe read&r. rThfe :inveatig*fcors do rmfentioft .that 

'~->-vv^ v • , ; ) -~im^j> « •* \4* ''~ ' :V: V- y * c •*• . 

difc£:fc^ y#ajt±%ily. tt : aB .u^d in 

y . f *■ ■' ' d" * i >: r* .*.»'.... "V . i' ^L' > •'••• T • • • ■ - . 

this, study, wa^^ the 
Results and . Arial^ is, Significant .demon- 

strability by the teachers be observed 

by a panel o£ judges." " fi/f ^ ,-•*»- 

"' / . ' • . ' ' ■ 

Second, it is not clear just how the 21 teachers were chosen nor do^ 
we know anything about the characteristics of the group, e.g., 
scietace subject areas, number of years in teaching, ^arge school or 
small school, etc. The /investigators indicate that the subjeats to 
be interviewed were selected /by a stratified random sampling ttech- 
nique but they do not indicate the size of the group from whiph 



.the 21 were selected if a larger- group was nominated by the nominat- 
ing teachers* I can only Infer that such a group wasV^omina ted since 
the 21 teachers used ^.n the study represented a sample . Although 
each of the 21 teachers interviewed was identified as being highly 
competent, the criteria upon which the nominations 'were based, if 
. other than individual priteria, were not reported. The inyestigators * 
also, failed to make clear whether the 201 randomly selected teacher/ 
nominators were all science teachers, ' * . ' 

Third, the mechanics of the teacher interviews were not clear. The 
"reader might reasonably ask: * 'tWho conducted the intetyiews?" Tt Wej:e 
* the interviews structured or uristructured?" 1t What werfes the criteria 
for ranking the competencies?" Since the final list of 12 generic - 
competencies was derived from an analysis of the 21 interviews and 
submitted to the teachers for ranking, there is a ^question in my 
mind as to why the investigators bothered. with asking the teachers 
to rank the self-generated preliminary competencies during the inter- 
views. The criteria. for ranking the 1 2 g eneric competencies were not < 
reported by the investigator^. 

Fourth, it is not clear .how the judges who rated the videotapes were 
selected; but more importantly the investigators ^ail to report whether 
the judges #eceived any training in identifying and rating the compe-" 
tencies.. Cohsequently, one'does not know the extent to wtfich the 3 
judges agreed about classroom exemplars of the competencies prior. to 
actually gating the videotapes. 

Fifth, with regard to the v±4eo taping, the investigators fail to report 
whether the taping covered a variety of settings, e.g., lecture, pre- 
laboratory, laboratory, etc., and place the lesson sequence, e.g., 
beginning, middle, and end. If the 10-20 minute tapes were made only 
at the beginning of a lesson and only of lectures, then one might 
reasonably expect that some important science teaching competencies 
would not be d^Sbnst rated. 

Sixth, the procedure for rating the competencies is very sketchily 
reported. The investigators do not indicate whether the Judges viewed 
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the videotapes together ot separately separately, was the order ^ 
of the tapes changed for each viewer? Furthermore, the unit of 
analysis, e..g., time, sentence, phrase, ^etc. , was not identified 
for the raeder. 

, -7 

One must, also, ask about the frequency counting procedure used. The 
investigators reported that the demonstration of competency 1 Qcnow- 
ledge of the subject) was rated as ipuperior 63 times, satisfactory 
162 .times, and poor 36 times. The investigators state, "In other 
words* the competency was judged as demonstrated 261 times (63+162+ 
36 ■ 2wL) .." One n^ust ask if each of the ratings was a discrete 
instance of behavior — in which case the observations could be summed, 
or was one judge's "superior 11 rating of a competency-another judge's 
"satisfactory"? If the latter k is the case, then, many observations 
of a competency may ahavd been counted more than once which in turn 

raises concern for the reliability of the raw data. Furthermore, 

* ■■ • • • 

how does one v count 33 non-demonstrations^of a behavior? How can one 
count k competency 'if it is not (demonstrated unless it is made clear 
to the ratet that a certain behavior Should occur under certain cir- 
cumstances and counted as a non-demoi^stratioto when it does not occur; 
if this was done, the investigators did rjot say. so. In my opinion, 
the reliability of the judges' ratings is quite questionable, 



Seventh, it' isj^heoret^cally inappropriate to use a ranking statistic, 
i.e., Kendall's Coefficient of Concordance, for measuring agreement 
among fudges' ratings . * In effect, the use of this statistic builds 

a high number of ties wherie one has 14 judges, 12 competencies, * < 
and only 4 possible rankings (ratings). If such a statistic is used, 
then, tl^e investigators are obligated to describe what happens to the 
sampling distribution when a high number of ties are bu'ilt into the 
matrix. 

Finally, in reporting the results, competency 3 (individual abilities) 

* 2 ' 

was left out in the initial discussion of the X results but mentioned 

j v ' . * ■ 

later in a restatement of the findings, . 
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Ttye inveati;g&tors have chosen a. worthy, albeit complex 1 subject for 
'their research but have confounded whatever might be of value by 
vague and incomplete reporting, I am especially concerned about the 
reliability of the judges' ratings due to the possible, and I th£nk 
probable, lack of discreteness of each rating and the use of Kendall 1 
Coefficient of Concordance to test inter-rater agreement. Therefore, 
the statistical base for determining the validity of the competencies 
(as defined by the investigators) on which the investigators \iave 
built their case is highly questionable. The competencies do have 
face validity, however; in fact, the 12 competencies identified in 
the study have been rather commonly accepted objectives of science ; 
teacher education programs for .some time. 

Finally, replication of this stu4y would be impossible if one uses- 
only this report. Ill-defined terms and , vague research procedures 
will only lead to the production of similarly confounding results. 




zarovitz, Reuven and Addison E. Lee. , "Measuring Inquiry Attitudes 
of Secondary Science Teachers." ' Journal of Research in Science 
Teaching , 13(5) :455-460w 1976. ... , . v . . 

- Descriptors — Attitudtes; Educational Research; *E valuation; 

♦Inquiry Training; ^Measurement Instruments; Science Ediica- . 
' tion; Secondary Education; ^Secondary' School Science* 
*Teacher Attitu^e^f :,i y \ 
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Lazarowtiz, Reuven.) lf Does Use of Curriculum Change Teachers 1 Atti- 
/ tudes Toward Inquiry?" Journal of Research in Science Teaching > 
13(6) :547*552£ 1976. 

Descriptors — -Changing At ti. tudes ; *Curriculuni£ Educational 
V Innovation;* *Educational Research; * Inquiry Training; 

Science Education; Secondary Education; *Secondary School 
Science; *Teaeher Attitudes . 
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^ Expanded Abstract and Analyst Prepared Especially for I.S,E, t by 
Rog£r G. Olstad, University of Washington^ 



Purpose 

The purpose *bf these studiq^ was to develop^^^instrument intended to 
measure inquiry ajyfci^ ides of secondary school science teachers, and, 
1 using this instrument , to investigate the effect of curriculum usage 
on spience teacher attitudes toward inquiry. 

* \ : '" - V"' /• • ■ _ . * 

^ It was hypothesized that science , teachers tohb used "tiew? prc^JgrStns 

(defined as BSCS, PSSC, HPP, CHEMS, CBA, and ESCP) would hold a mote : 

favorable attitude toward an inquiry approach to science teaching 

than science teachers who did not use these programs. It was further 

hypothesized that there would be a u relationship between the number 

of years that, the "new" programs had^been used by these teachers and 

their attitude toward an inquiry approach to science teaching. 

t 



Rationale 



■ ¥ 



These studies assumed that Effective implementation of the "new 1 
curricula required science teachers to use inquiry Strategies. 

30 



Studies were cited which indicated that there were variations in 
the implementation of these programs' inquiry-approfeh base* (Amos, 
Dillon, Gagne, Parakh, Watson) , that these variations were often 
jlue to teachers 1 personal traits and philosophies- {Black, Gallagher, 
Montague & Ward, Sadler) - 9 that mere use of inquiry science curricula 
did nc(t assure a positive philosophy with regard to the use. of tf 
inquiry* strategies (Barnes, Gallagher, Montague & Ward) , that 
training did positively effect teacher use (B^rries£ Kochendorf er , 
Ost), and that tfiere waf^a relationship betweeA the amount of teach- 
ing experienc^ and' the effective use o£ these programs (Blankenship, 
Barnes, fcochendorfer) . Most of these studied were conducted prior 
to 1970- \ . - 



^search Design and Procedure (Instrument development) ^ 

-The instrument, known as Inquiry Science Te$£hing Strategies (ISTS) , 
was purported to measure inquiry attitudes of secondary school % ) r _ 
science teachers.' Eighty-three items were initially developed with 
reference to three areas: classroom teacher-student interaction, 
laboratory investigations, and textbooks used. The ijtenjs were 
selected according to their relationship jfto inquiry teaching which 



was denned including gtudent . forniulatibn of ptoblems 'and hypo- 
theses, student designed "studies involving data collection and 
interpretation, and student acceptance of different results for 
discussion and interpretation in a learning environment whei^the 
student is actWely participating and the teacher is student- 

oriented rather than subject-oriented. ' 

* * 

A 

Half the statements were positively worded regarding inquiry, the 
others negatively worded, and they were set up using a typical five- 
point Likert scale ranging from strongly agree* to strongly disagree. 
Procedures involving a panel of judges (seven professors of science 
e^Ktcation) and a group of 30 secondary school science teachers 
reduced the. instrument to a total of 40 items and established its', 
content validity. ' . ^ 
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Construct, validity was established by the technique of known groups 
by obtaining mean scores from five groups to whom the instrument 
was administered. 

Group 1: * 7 Professors of science eduction 

Group 2: 16 Graduate students in science education V 

Group 3: * v 47 Inservice secondary science teachers ~~^^3r 

having had a 4% day workshop in inquiry 
methods 9 

Group 4: .. ■ 44 Preservice secondary science teachers in 

: their first day of class in the methods \ 

course * 

Group 5: 735 Secondary science teachers from various 

cities in Texas * 

Instalment reliability was computed ysing Kuder-Richardson Formula 
20, VieXding alpha coefficients of internal consistency for each of- 
the groups. , * , 



yinding^ x (Instrument Development) 



The mean scores on the instrument by group were .as fallows? 1) ■ . *. . 
181:57, 2X 160>68, 3) 146,82; 4) .145^6, 5) 143.27. ' TV authors 
stated that, * "as is to be expected when using this technique (of 
known groups), the meaii score of each group differed according to 
the level of involvement in science education — i.e.* the more" the 
involvement, the higher the mean score." 

Tests of significance between pairs of group means showed that Group 
1 differed significantly from Groups 2, 3, 4 and 5; Group 2 differed 
significantly from Groups .3, 4 antf^S; and Groups 3, 4 and 5 did not 
differ signif icantly from each other. 



Reliability (in alpha coefficients) was .54 and .48 for the first 
two groups, and ranged from .69 to .85 for the latter three groups! 
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Interpretations (Instrument Development \ 
v. ' ' ». - 
/ *The authors^ concluded that the 'results support the use of ISTS in 
\^ determining the inquiry attitudes of secondary school* science 

teachers. They further suggesteS that' ISTS made possible a compar- 
/ . ^ / ison of the degree of acceptance of th^ inqjuiry^pp^^/h^by the ... 

individual teacher in relation to known groups and could be used 
»by supervisors or methods course instructors to recognize areas 
in wfeich teachers need help. VThis recommendation was b^sed.on the 
construct validity of ISTS and the high alpha coefficient ( -85) for 
Group 5, the roost heterogeneous population studied. 

- f- - . :. . ... • _ . ■ . . \ ., ■ • v 

- • "■■ ; o 3 ' 

. ^ Research Design ai\d Procedure (Attitude Change) K 



The JSTS and a personal data form were distributed (by .district 
science supervisors) to an unstated number of Texas secondary school 
science teachers- A total of 507 responses were returned to the 
supervisors, for analysis./ These teachers represented 30 Texas commun- 
ities, and most secondary school science subject and grade level's. 



Findings , (Attitude Change) 



It was found that biology, chemistry, and earth science teachers 
who useSl the "new" programs had significantly more favgrable atti- 
tudes toward inquiry strategies - than did teachers of those subjects 
not using these programs. For physics and life science teachers 
there was no significant dif^rence between users of "new" programs 
and those not using them. » * • ' 
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When compared by years of teachipg^ expedience, experienced biology, 
chemistry, and earth, science teachers using "new'f 1 programs had 
Significantly more favorable attitudes toward "inquiry strategies 
than h^d^i ess expefienced teachers of these subjects. Nq signifi-, 
cjgnt relationship was found for physics teachers, and there were 
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not enough life science teachers using "new"* programs to analyze 
• by years' of teaching experience. 

.,. .,, .... , , \ - 

The author also analyzed the results oh 10 specific itsgnjs. (no 
rationale provided Sor the selection) of tiie inventory, comparing 
teachers using,, "new" programs with non-users. Half of the selected 
users^and non-users were significant. It .was -also- found that the. 
degree of rejection of negative items was lower than the^iegree 
of acceptance of positive items. /• 



Inconsistency ;of ISTS ranged from .ft t 



Coefficients of lnternar*cbnsis^ency ;of ISTS ranged from .ft to .81 
for users and from .66 to .86 fot; non-users. 



Interpretations (Attitude Changed * 

* ■ • • 

^ The, author Stated that "the results of this study show that secondary 

science, teachers who use new programs in their teaching activities 
more favorable' attitudes toward inquiry strategies than non- , 
users, and that years of experience in the use of new programs is 
related, to more favor-able attitude^toward inquiry strategies. 
However, it seems that thi'sHs not a* tiniversal picture. There" is \* 
no assurance that for each particular element of an inquiry approach 
teachers^^wlll hold' a^favofable attitude to the same degree. There 
are no inquiry or noninquiry teachers. Neither .the use of , new > 
programs rior the length of time they are used assure proper inter- 
\ pretation of the inquiry approach advocated by the programs. The 
results suggest that teachers need more training in specific com- 
petencies of elements of inquiry activities...." - 

ABSTRACTOR'S ANALYSIS ■« 



The attempt to develop an instrument-designed to measure science 
teachers' attitudes toward inquiry Is commendable. " Whether this 
instrument measures this guality depends'upon what is meant by 
"attitude." The authors, provide np definition of theirs and the 



reader is left with hiw/her own understanding ,of that wprd.*~For 
this, reviewe^^jdef inition which has been found usfefuX is "the 
difference between knowing what to do and dpinfc it. 11 For example 
(using ^ne of the items from the ISTS), know ing 'that "students 
are ofjten capable of designing valid experiments" is different 
than being skillful in structuring lessons involving studetit 
designed experimer^sTSmd both of these are different than using 
lessons' C^olv\nj/ student, designed experiments. It is in the 
latter^ase OTCe gains an insjLght into th^ teacher's attitude 

regarding student designed experiments. 




( : ' 

Thus, /whten the authors write that "ISTS matfes possible a comparison 
of the degree of acceptance of the inquiry approach," it should be 
clear\that what is meant is intellectual acceptance and not neces- 

rily\kcceptance in teaching practice. 'Further, when the author 
states thkt the results suggest that teachers Should hav£ more£ 
training in "specific competencies," one should note that training, 
in a competency is nis£the same as changing an attitude, although 
.the one might lead to che other. (The research results whicji 
suppor£ the call for inore- training in specific competencies were 
not reported in^ the two papers reviewed.) * 

The development of £he ISTS instrument and the interpretation of 
its use would have been greatly enhanced had the aythor(s) examined 
the relationship of teacher- scores to teacher practice. ^ While the 
authors were careful to develop the content and construct validity 
of thdUr instrument, the most import anjj^ind of validity for such 
an instrument is its predictive validity. With a high degree of - 
predictive validity, the user (and we readers) would have some 
confidence in what the instrument was really measuring, e.g. know- 3 
ledge about Inquiry or practice involving the use of inquiry; What 
tlie obtained scores predict conclrning practice remains an unanswere 
but .vital question. 

* * 

The title of the second article ("Does Use of Curriculum Change 
Teachers 1 Attitudes Toward Inquiry?") poses a Question which is 



never answered, and the research design used does not allow for^its 
answer. The support for 'the h$j$othesis that teachers (of only some 
science subjects) who use "new" programs have more favorable atti- 
tudes toward Inquiry strategies does not necessarily mean that the 
use of the "hew" programs ^caused any change in these attitudes . 
It is equally plausible that these teachers already had more favor- 
able" attitudes , and that these more favorable attitudes toward 
Inquiry influenced their selection of curri*cul£, resulting in 
their use of the "new" programs. 

As for the relationship between the number of years of teaching 
experience of users of "new" programs , and their more favorable 
attitudes toward inquiry strategies, this could be explained in a 
number of ways in addition to the author f s assumption, th^t ''teachers 
who teach the new programs "will ^develop more favorable attitudes ^' 
toward inquiry....," For example, it could be^ft^t longer users of 
"new" programs had to have more favorable inquiry attitudes to 
stick with the programs tha* long; Again, the authors findings 
do not, necessarily support' his conclusions/ . " 

^ ■ ' ""■ -•' / /.. ' > ^ '. 

The author offers no "explanation* for hisfinding that biology 
teachers 1 attitudes were significantly different between users 
and non-users, while for life Science teachers there was no signi- 
* ficant difference, or the finding of no difference in attitude for 
f physics teachers, whether users or not. These "discrepant" find- 
ings must be accounted .for if one intends to buil^ a generalizable 
model regarding Science teachers' attitudes toward inquiry 
strategies. (Perhaps the non-significant findings for physics 
teachers could be accounted fo|T by the fact that the initial 
selection of the item pool was largely from biological sources.) 

The assumption that 44 student teachers in their first day of class 
in a science methods course have a higher level of involvement in 
science education than 735 experienced Texas science teachers is 
questionable. Yet, the authors state that the higher mean score 
for the student teachers Vis-a-vis experienced science teachers 
' V ,■' x - * 1 
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Is to be expected because of their greater involvement ,\apd' upon 
this assumption rests the determination of. the instrument f s .con- 
struct Validity, i ■ " 

The recommendation that ISTS be .used as a diagnostic instrument in 
which a science teacher supervisor looks at an individual teacher's 
responses to individual items* on the inventory and proposes assis7 
tance in areas where attitudes are lower is aft ^approach which few, 
if anyjT science supervisors would find workable unless an extra- 
ordinary supervisor-teacher relationship had been previously 
established. Indeed, in these studies, it might be asked to what 
extent were* the responses (were they anonymous?) affected by the 
use of science supervisors to distribute and collect the instruments? 

Finally, for the author's stated conclusions in the second paper 
(quoted previously) , it is left to the reader to interpret the 
meaning of what is written, antf to compare this with the findings 
as cited . ^ 



The relationship of science teacher attitude to practice is impor- 
tant. Future research should take into account research already 
"done regarding teacher attitude and its measurement, much of it 
generic in nature and not pert^j.ning to science teachers per se. 
Further, the v predictive validity of attitude measures must be 
attended to by serious researchers in this area. 
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Purpose 

Barufaldi et al . investigated changes in attitudes held by preservic,e 
elementary education majors toward inquiry techniques that are used 
as part of teaching strategies for elementary school science programs. 

Rationale 

^ * 

The investigators gave examples of inquiry skill development activities 
for elementary school silence programs and emphasize the role of 
teacher as stimulator/ -The research of Allen (1970), Bruce (^971) and 
Smeraglio and Honigman Q.966) was cited in support of behavioral 
changes in children exposed to stimulating teachers. Although not 
stated, the inference was made, that the direction of the teacher as 
Initiator of inquiry activities plays a significant role in causing 
desired student behavior and therefore the attitude of the teacher 
toward the mode of learning should be 'known. 

Research Design and Procedure 

Barufaldi et al . conducted a pre- posttest investigation of attitude 
change, using an elementary science methods course as intervening 
treatment. The course was stated po be an exploration of facets of 
modern elementary science programs. The variable being treated wa£* 
thte set of attitudes preservice teachers held toward inquiry techniques. 




The researchers used a previously modified form of the Inquiry Science 
Teaching Strategies (tSTS) *as the test instrument. The ISTS is des- 
cribed as containing 40 items, half stated positively (inquiry 
supporting) and half negatively* (non-inquiry supporting). Subjects 
responded on a Likert-type scale to each statement. A possible score 
of 200 is indicated. Previous validation yielded reliability scores 

* of 0.41 and 0.83 by Program TESTAT (an item analysis^ and a Cronbach 
OC coefficient of 0.96. 

From an elective elementary science methods class of 146 students 
(fall of 1973), 74 students were given the ISTS on the first day of 
classes (pretest) and the Last day of classes (postt.est). 

Mean scores for subjects and fo^ seven test items were compared using 
one-way analysis of variance. The item analysis was used to check 
consistency in attitude change. 

■ ■ \ ■ * < 

Findings 

The investigators found the pretest mean score to be 146.14 and the 
posttest mean score to be 155.41, a difference significant beyond the 
0.01 level (F ratio 77.54). 

Six of the seven items also showed changes significant beyond 0,Qland 
the seventh was significantly different^ beyond the 0 t Q2 level. The 
researchers noted that change was evident in 13 of 20L items (65 
percent) in each of the positively and negatively expressed groups of 
iteojs, * 

• Interpreta t ions 

The authors concluded that the subjects had more favorable attitudes 
toward inquiry strategies following completion of the course. 
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ABSTRACTOR'S ANALYSIS 

J 

Many educational ^investigators seem bound by unfortunate research * 
limits. Unyiela-ing parameters also face the physicist who wants 
to be free of earthly gravity. Nonetheless, all investigators should 
strive for a purity of setting. The research should be true to ,the 
cause it addresses, and bias must be absent or recognized. 

»'••. t ■ ' - 

Educational investigators would seem to have mo^e difficulty in over- 
coming |>iases connected with the research setting than ,is 4 desirable. 
The shortage of funds for ordinary research forces the choice between 
ideal and possible. What can be done is done and sour grapes have to 

I 

be eaten. The results, however , may be open to too much criticism 
and yield too little that is fresh and useful. To- gr^sp—gassible 
research topics may mean using subjects at hand, either those who can 
be persuaded to accept^ a benign treatment or those under the "control 11 
of the investigator. If the latter, they likely come with certa^ft 
mental- sets, prepared if not^Mcessarily positive. They may be aware 
that part of the routine is completion of an answer sheet, and the 
instructor happens to teach certain concepts that are within the 
question-answer context of opening day. j , 

The foregoing is meant to be mo*? realistic than cynical. For it is 
held to be realistic that a response pattern is stimulated, Is likely 
to be expected, aind that self-selecting subjects will cooperate .so 
that concomitant testing will reveal the mental set. A negative^or 
neutral resul^, on the jbther hand, would be an unwelcome outcome for 
a teacher-investigator and an indication of the worth of his methodo- 
logy. ' 

' / . > 

The analysis need be little more ' than shallow. Since the turn of the 
century it has been shown thlat investigations of classroom techniques 
will yield desired results regardless of the investigator, his method 
of testing, and the methodology within his program of instruction. 
Barufaldi et al . may be add£d to a long list. 
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None of the parameters of- the spudy indicate alterations in concept, 
methodology, or research design from previously reported studies. 

\ 

Conments made above about the setting of educational research deserve 
more specific exploration here. Texts on" research design caution 
investigators about sample size and interpretations that flow from 
research. The cautions should be heeded, and possibly journals 
should tighten their editorial policies to exclude the obvious arid 
the limited findings. There seems room for criticism with respect to 
the research under review. At best, it seems that 74 selected 
students (out of 146 — no .explanation given) were* successful in receiv- 
ing the message that inquiry teaching strategies are "good" and 
lear liable, or that the attitude set is. 

Most educators should not be surprised at t the outcome and should 
wonder what further research will follow and why. It may be of mor^ 
use to identify the non-learners or recalcitrant students and weed 

v them out of a program that requires inquiry skills -and allow their 

;■ assignment to other roles in life. 

Too many article^ (this may not be one) flow from doctoral study, 
bearing the names of several persons, reporting the obvious. It 
may be time to stem the flow of reports* Even more drkmatic would 
be the upgrading ^ research in this area of investigation. Perhaps 
attention could be diverted to the environmental factors that influ- 
ence the success*of graduates in implementing their positive atti^ 
tudes at the classroom level. 
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Purpose x 

The first two authors 'of this article devieloped a written format pack- 
age to teach preservice and inservice elementary ^bachers^ how to use 
inquiry and process-oriented behaviors during science instruction. 
This study was subsequently evaluating the effectiveness of the model*^ 
by: 1) comparing the trained teachers * ability to use process skills 
as measured by paper/pencil/test items with an untrained group* of 
teachers, and 2) demonstrating an increase in the use of indirect 
teaching behaviors by the experimental (trained) teachers, 

- • ■ ? ^ ■ • 

Rationale 

Cutbacks in funding for preservice and inservice training in science 
education for elementary teachers raised the need for a pi;int format 
package of instruction accomplish the same goals. Few elementary 
teAchers aire adequately trained in science content, and fewer still 
receive any special methods instruction specific to the unique effect 
and* processes required for effective science teaching. Thus the ^fl& 
of the NSF institutes and money for consultants needed for revamping 
science instruction and g^hancing the content expertise of elementary 
teachers represented an appropriate rationale for undertaking instru^ 
tional development* of the inquiry/process 'skill s^jiagkage. 

i ; •■ a, * ■ . 

The study is obviously an evaluation effort being r epor t ed ^ncter a 
research label. If the results were used for revision! of the package, 



then jp^e rationale for the study would fce strengthened as one of 
fdrmative research. 



Research Design and Procedures 

A prerpost test quasi-experimental design wals employed for the first 
question. The equivalence between the two groups of teachers was 
determined only by pretest score's for criterion variables. Subjects 
In the experimental group (N= 70) were ^enrolled in either preservice 
or lnservice science methods training while control subjects (N=32) 
were enrolled in a general curriculum course. 

The instrument used to determine -treatment effects for inquiry/process 
skill attainment was designed by the developers using a modification k 
of the AAAS-Sci^nce Process Megjgure for Teachers, Form A , 'The modi- 
fled instrument has a Cronbach alpha coefficient of 0.86 (N - 23) and 
a test-retest reliability coefficient (with a seven-week * interval) of 
0.48 (N * 32). Tl\e' hand-scored process skills tests based on a rater 
protocol yielded aft, interrater reliability of 0.98 (mistakenly reported 
as interforeter^ reliability) . * 

The second question was addressed with a change score design using the 
Instrument for the Analysis of Science Teaching (IAST) Version 2 (Hall, 
1972) to collect interaction data during the first twenty-minutes of. 
each class. ■ The first two authors served as observers and had inter- 
observer reliability^oefficients of 0.82/ 0.79 and 0.84 (Scott 
Coefficient Pi) prior to coding the experimental teachers 1 pre- 
treatment behaviors. z ' 



obs^ 



Teachers were obsjefrved and then exposed. to the packaged instruction, 
either singly or in small groups, and performed all the activities in 
each unit. Upon completion of each unit, teachers were to modify the 



jnlt 



activities and present them to their elementary students. No mention 
was made of grade level assignment for these teachers, and it is assumed 
that 'all grades' were represented. 

44 * 



Teachers were observed and their interactive behavior again coded for 
eq&h unit. The developers provided some unspecified form of "help" - 
in getting the teachers to use the inquiry process during the first 
« unit but this assistance was "faded out" during subsequent units. 

..' . , • . ' ' * '. \ 

Outside of classroom teaching time, teachers spent fubm 18-24 hours 
. " . v. \ - . - . 

interacting with the instructional .package. At no tis&p did the 

1 cJptvt^ tteachers read or interact with the materials, and, it is 

assumed although not .Specified, that the control group <Tid no 

teaching of science or any other content areaf before being posttested. 

Before analyzing between group dif f erencesyon the skill achievement 
question, the pretest scores were analyzed using a one-way ANOVA to 
show equivalence on the nine process test scores. Sirfee two tests 
significantly differentiated the groups, the^ pre-posttest analysis 
included a one-way ANOVA for the seven equivalent process\tests, and 
an analysis of covariance was usred to compute the F value for the 
other two. 

4 ■ 

V * 

Analysis of 'the data collected by classroom observation yielded a series 
of eight indices (five teacher behaviors £nd three student behaviors). 
Correlated t' statistics were use^fts^dirfpare pre-an3 post treatment 
behaviors. ■■' ' 



Findings 

The trained teachers Outperformed the control group (p4.05) on five 
process measures (observing, classifying, predicting, controlling 
variables and the cbmo/dsj/^e score). Measuring, communicating, infer- 
ring and identifying' variables were not different between groups 
although inferring/and identifying ^variables had been different on 
the pretest. Wh/en the means were adjusted us£t)g an analysis of 
covariance withr the pretests as covariates, the experimental group 
was also higher on^the process score for identifying variables. Gain 
^cores were also signif icantly greater fdr, the experimental group (p 

,^•05) on seven process skills (all except measuring and communicating). 

'' * . v 
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Teacher behaviors which were significantly modified by treatment with, 
the Inquiry package including lower ratio of closed to open qyestions 
and lower percentage of continuous lectures. The other behaviors were 
unchanged* 

* r ■ ■ ■ - 

A post hoc investigation of trhe questioning behavior using Galla^ier f s 
questioning categories (Nelson^ 1969) yielded data which wer^ analyzed 
with a correlated t statistic. /This analysis revealed *hat, after 
Interacting with the instructional package, the teachers reduced ^ 

^ their use of cognitive memory questions and significantly increased 

* > . * .. . r 

their use of divergent questions. *\ A 

Student behavior also^changed^sjlgnif icantly after their teachers had 
studied the units. Students exhibited a significantly more positive 
attitude toward^s^ence instruction, were involved in a proportionally 
1 greater amount of non-verbal activities and mor.e peer interactions, as 
measured by the interaction analysis instrument ClAST v. 2). Addi- 
tionally, correlated t statistics applied to pre ^nd post treatment 
data revealed significant increases in the frequency of- student group 



activities and peer interaction during both small group and total 
class discussion. 



\ 



Interpret a t ions 



e authors suggest there' were incongruities between the package 
strategies and the evaluation instrument: For example, the package 
emphasized creativity in devising measuring techniques while preci- 
sion and accuracy were primary criteria on the test which* resulted 
in no significant gains in the treatment group's measuring skills. 



The instructional package was judged effective and successful in 
changing teacher behavior from lecture to inquiry mode #/ 

If teac^rs are to develop skill in teaching scientific processes 
using an inquiry approach, they must receive specific instruction 
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toward those goals. There is a real need for Instructional packages 
In print format such as this one. 



J 

ABSTRACTOR 1 S ANALYSIS 



The failure of the authors to se$ the problem of inadequate science 
Instruction as not only one of Inappropriate preparation in^ teaching ' 
effectiveness fekills but also one of inappropriate materials and » 
Instructional strategies is evident in their design. With inquiry 
oriented instructional materials $nd strategies, teachers will use an 
inquiry /process oriented approach and will use more indirect behaviors, 
ask more questions and have students who behave more independently and 
achieve higher over a long range of instruction (Egelston, 1971). In 
their design contrbl group subjects were never allowed to teach the 
content. It is therefore unknown if interaction with the instructional 
package (treatment) accounts for the between group differences, or if, 
teaching the content itself would affect, criterion scores, 

Hissing information was the nature and extent of the assistan.ee in 
using an inquiry approach provided in the first few units and then / 
"faded out." Indeed, this technical assistance could account for the 
gain scores, behavior changes and between group differences. Documen- 
tation of the necessary assistance would be the best formative evalua- 
tion *for this' instructional package. 

The report was well written and the study was thoroughly done. In 
particular the post -hoc questions revealed additional detail which 
""helped ^ clarify the results. 

It would have been helpful to know the authors' perceptions of the 
limitations of; th$ category system. Other systems of interaction 
analysis have been developed ^nd used in science classrooms , and 
the choice of the IAST should have some rationale. 



Finally, the Implication that evaluation items for the criterion 
variables did not -match the strategies in the units lead/ this 
reviewer to advocate revisions. There was no indication that revi- 
sions were planned. 
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Purpose V . 

The purpose |Of- this study was to determine the instructi6t?al priorities 
which have emerged tn the implementation of the Israeli -BSCS Adaptation 
Biology Program., The authors distributed a questionnaire to a random 
sample of^teacherS and another sample of their students to determine , 
the prior it^es-^nd frequency of occurrence of nine .^selected instruc- 
tiortal practices .\ Table I li^s the instructional practices and Table 
!■ ,11 outlines the study and the" various comparisons which were utilized. 
The ^irst comparison involved the contrast between how important the 

teaqhers* viewed the instructional practice (priority) with -their perdep- 

* * 

tion of just how frequently the practice was utilized. ,The second / 
comparifijpn involved the student's rating of each practice with his or 
hier perception of how frequently it occurred. A third comparison con- 
trasted the priority ratings of the teachers arid those of the students. 
*rhe« final comparison involve^* the contrast of the repotted relative fre- 
quency of use of each practice tiy the teachers ^hd their students. The 
contrast between whatsis important and how often it occurred w$s the 
central theme of the study. 



Ratio nale * * 1 V * 

- - '.- ' • * 
'This study is one of several interconnected studies which examined the 

priorities whicfh emerged for teachers and their students. % Previous 

^ studies examine*!* the expectations and the objectives of randiSfn samples 

of ^eac'hers and students. In a sense it is the emerging methodologies* 

j ■ ■ ■ ^ , . ■ * . » ... 

whicli are examined and analyzed. # ^ % 



The study used a 22-item two-part questionnaire to obtain, the data. 
The first part of this instrument utilized a hine-item, five-point 
Likert scale which asked the participants to indicate how important 
they considered each of the instructional practices. 

The second part of this instrument used a five-point scale which 
required the participant- to indicate how often the various practices 
occurred in their classrooms. For example both groups were asked how v 
important they felt field trips were in this course'* and thhn asked how 
often did they go on such trips or utilize such trips for homework , 
assignments. 

The authors reported that the instrument had an alpha (Crohbach) 

.reliability which varied between r=» .72 to .87 with the different 

samples. A stratified random sample was used whicfi included city 

academic schools, fural kibbutz schools and agricultural "?school6. 

PuestiQnnaires were sent to a" random sample of ^0 teachers and 66, 

replied (82 pe^fent^re^ponse rate) . The authors, a'lso asked the 

teachers if they would administer the questionnaire to their students. 

Twenty-four teachers did comply with this request (36 percent response 

v " * 

rate) and a .total £f 624^tenth grade students (age 16 yeaA) completed 

the .questionnaire. Ohe assumption underlying this procedure ts that 

e data from the 24 teachers can be generalized to the 66-teacher 

sample. The authors did compare the results of the questionnaire of 

these 24 teaches with the larger group and found no differences. 

The st atist ical method utilized in this study' was a series of t-tests 

between- each sample's priority and frequency of occurrence for eich of 
the nine instructional practices. 

' ' 3 •• ' : . • ■ ; > • , 

Findings ' , • ■. ' *~ 

- ■• -V ■■ ' ) ■ \' . • • • 

1. Students^ saw the text and its questions and exercises as mote impor 
tant than di&the teachers. Jhe students also reported higher 




levels of use of these' items. In. terms of the t- tests, '.the 
students rated the Importance o£^the text significantly higher 
than the frequency of use. (Although both ratings were in the 
high ranges) 

2. For Interviews and securing information from other sources, *■ 
teachers saw these activities as more important and occurring 
more frequently. The disparity between the rated importance 
and occurrence of other sources was significantly higher for 
both groups. Both groups prioritized this^act ivity but both 
reported much lower levels of occurrence. The same finding 
was found for students with interviews. They valued this 
activity highly but experienced such interviews much less fre-? 
quently (.001 level). 

* ■ - * • 

3. Teacher!? found radio, television, lab and field trips more impor- 
tant and had relatively higher levelg of use than students . Both 
groups indicated significantly higher priority levels for these 
activities than actual occurrence. 

- •■ * 

4. ^ teachers rated original assignments and individual investigations 

as more important and also indicated higher .levels of use. B otti 
groups had significantly higher levels of priority for individual 
Investigations; than for its frequency of occurrence. 

Intrterpretat ions % * - * * t * 

1. The authors concluded that the*disparity\>etween teachers and 
students' about classroom events was expected and- was useful to 
the implementation team. 

2. The authors reported a high level of/use for many conventional 
Instructional practices despite the innovative nature of the 
curriculum. \ 

• 1 . ■ ' * 

i 

3. Lab was perceived as-valuable and used frequently by both grotups. 
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4. The two practices involving originality and doing individual 

assignments and did seem to have the level of occurrence that tt\p 
team expected. In fact, the low level of occurrence of these 
items should have implications for the developers of this cufijic- 
ulum. 

% . ■ ■' : ' ' * ■ 

ABSTRACTOR'S ANALYSIS 

This study has several im^orjtant strengths. First, it is a serious . 
attempt to utilize formative evaluation in the large-scale impIemen-£Jv 
tat ion of an innovative curriculum project. The study is one of 
several done in comparing the perceived priorities of teachers and 
student 8 with their perceptions of the frequency of occurrence o£ 
related practices. Thus, the priorities which emerge are examined 
both by the students and ttfeir teachers and both viewpoints are 
contrasted. #y doing several related studies it is possible to use 
each one as a point of reference to the other and arrive at more 
useful conclusions.^ 

In terms of design, the study utilized a stratified random method^ in 
selecting the sample which has to be a strength. The authors rejected 
the use of the Flanders observatior^/sys tern because o^the difficulty 
in using it with a large random sample. This prqcedure would jseem 
-justifiable particularly when the* researcher is not concerned with 
the affective climate .of the dialogue. • 

One weakness in^design which did emerge was the failure to utilize a* 
control group or stratified series of control groups. Since the 
authors were interested in the implementation at three different 
settings, then it would* have been profitable 'to have control groups 
in each setting. Without,, such baseline data it is difficult to 
judge the effects the new curriculum is having. These effects are 
one of the primary reasons for conducting evaluation studies in the 
first place. By using a control group design the authors would have 
been able to compare the priorities of their- samples With samples of 



the traditional tenth grade biology classes. What if both groups 1 
priorities were the same? What if many of the practices turned out 
to occur just as infrequently for both the traditional and the new 
implemented curriculum? Without a baseline for comparison, none of 
these questions can be answered and it is not possible to- isolate the 
effects, of the implementation -process-. 

• : " , • . ' ' ' ■ 1 

Another design problem was the non-random nature of the student sample. 

■ _ 
By asking teachers to volunteer to te'st their students the authors % 

could not call this sample a randomly selected one. In fast, the 3 
percent response rate caused them to compare the teacher 'scores of 
these 24 teachers with the larger sample. Regardless of the negative 
results of this comparison we can still not gene^lizJP the student 
results to the larger sample with any d*g«e of certitude. The authors 
should have conducted a detailed analysis of why so few teachers cooper 
ated. Did a^iy one^ of_ the .three siiblevels h4ve higher percentages of 
respondents? - The better way would have been to randomly select stu- 
dents and send the questionnaire^directly to them. This procedure 
vdfckld have resulted in two rand ofii samples. 



/ 
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Another area closely related to design con&rns the instrument used 
in this study. This two-part instrument Was developed from an 
informal observation of the "transactions" the team saw during the ^ 
implementation of the program and also from the types of recommenda- 
tions they made to the participating teachers. Hhis procedure would 
assure that -the instrument had some degree of content validity 
(although the authors made no such claim); The instrument could have 
been strengthened by establishing other types of validity. It does 
seem possible that the frequency of the nine practices could have 



been observed in a small sample of classrooms and actual tabulations 



made of just how frequently each occurred. This data could -then be 

3 

used with the data produced by the questionnaire for both teachers 
and students. This procedure would isolate just how accurate each 
group's perceptions were' of these classroom events. 
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What if students or teachers were inaccurate by 20 or 30 percent? Is 
it possible that the students were more accurate observers than the ^ 
teachers? Such data are important if we are to make conclusions in 
relation* to perceived priorities'. 

In terms of reliability, the authors reported using an alpha reliability 
for each of their samples. This method of determining reliability 
uaWlly is performed on just one trial administration for each sample. 
2 Toe authors don't mention any other mechanism^ of determining the 
reliability of, the questionnaire. » One problem x£ith using only the 

** * 

rone administration for the subjepts is that the individual's day-tp=r 
day variations are not considered. Such variations do occur^for. many 
types of tests and are important to dontrol for in the construction of, 
any instrument. A test-retest method is frequently used to isolate 
this factor. It could have been used to augment the alpha reliability 
* which was used. By using the two procedures the authors would streng- 
then their claim fot developing an instrument which produced consistent 
results.' 



Another, more serious problem concerns the selection of items for the 
te^t. The authors don't mention any trial administration or any item 
• analysis in establishing tni ability of .each item to discriminate. 

In Part II most of the frequency data were based on just one item. 
Only three of the practices used two items to determine the frequency 
of occurrence." Perhaps the frequency of each practice could have 
been determined by Several homogeneous items. It is interesting to 
note that in one of the* three practices that contained two items 
there w6re no significant differences for one item and significant 
differences for the other. If both items were supposed to isolate 
the same practice, how can such difference occur? 

Another problem with the instrument was no mention in the research 
report of. the directions utilized by the samples. Th.ese data are 
always useful for a reviewer to <analyze. This reviewer had a final 
question about the difference between the affective part of the test 
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and the observational part. \In all the results the two parts are scaled 
on the same point value but the question arises: Are the two value 
systems really equivalent? Can one really^ equate neutral importance to 
some degree of frequency of one or more W the practices? Perhaps the . 
analyses should have remained separated for both parts. of' the test. 

Despite these weaknesses the instrument is a blend of high inference 
items (affective portion) and low inference items Cobservat ional part). 
Such a blendlng/Vas first recommended by Rosenshine et aL (1971) and 
must be considered a strength. * ' 

In summary, this report of a formative evaluation study shows the. state 
of the 'art of researchers in this area. The emerging priorities of any 
curriculum are important items to isolate an3 measure. Sopefully more 
and more school systems will utilize evaluation procedures to isolate 
the effects of each new implement-fed curricula. ^ 
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^purpose of this study was to examine the questioning styles of the 
r;f gar high school biology texts more Frequently used for college-bound 
'stuctents. Types of questions, including experiential and nanexperien- 
iial, frequency and placement of questions within the texts wetfe com- 
pared. , For the experimental questions the kind of inquiry processes 
elicited by the questions was examined. The hypothesis was that there 
would $>e no quantitative -differences In the questioning styles of the 
four texts. 



Rationale 



0 
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A basic assumption is that questioning by either the teacher or the 
text pl4ys an important role in eliciting inquiry behaviors Jji stu- 
dents. Research Indicates that the kinds of questions asked during , 
instruction determine the types of operations students perform. 
Because many teachers rely heavily on the textbook as a program 
guide, it is appropriate to examine the ability of in-text questions 
to prompt student inquiry. The authors state that little or' no 
research has been focused on relating textbook questions to the pro- 
cess of inquiry or to science teaching in general. Studies examining 
questions in written materials have not been applied to commonly used 
biology textbooks. 



Research Design and Procedure 

*The texts selected for the studylwe^e 1973 editions of the four mostb 
widely used high school bicrlogy tetfts. for college preparatory biology 
classes • The texts serveS as the independent variables. The depen- 
dent variables were the ratio of questions *to sentences per page and 
the frequencies of types of questions tallied according to the Text- 
book 'tjuestioning Strategies Assessment Instrument (TOSAI)^ developed 
by the Cooperative Teacher Preparation Project. A 10- percent random 
sample'^of' pages from each text was studied, - Erom each page afjcatlo 
of questions to sentences and the mean number of questions were com- 
puted. [ ~ *v . ^ 

The questions were classified as experimental or nonexperiential, 
which required the students to focus on phenomena not previously 
experienced. The . experiential questions were classified as rhetori- 
cal, direct information, focusing, open-ended . and valuing. The non- 
experimental questions were identified Recording to the following* 
inquiring processes: observing, communicating, comparing, 
organizing, experimenting, inferring and applying. Additionally, 
the questions were examined By placement in the text: initiatory, 
contextual, terminal and QaptionWl. % 

For analysis the questioning styles and frequencies were Converted 
to 'percentages for easy comparison among the textbooks. Only^the 
overall frequency of questions per page was statistically analyzed 
for significance and this was done by t-test. All pairwise con- • 
trasts for the four texts were tested. The particular questioning 
styles of a given text were compared but were not &nalyzied by 
statistical tests. » 1 ; 

The variables evaluated were frequency of experiential questions, 
placement of questions, types of experiential questions and patterns 
related to the science/learning f&qcess questions. Other differ- 
ences noted by the authors were placiemerit of the laboratory 
Investigations and the general distribution of questions within 

the text. For example, the Modern Biology text asked the most 

• * * ■ 
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questions in the first f ew ^apters^ Interestingly, it .was noted 
that the level of sentence complexity and reading difficulty were 
similar for all the texts except for the BSCS Blue Version which 
was moire complex. 



Findings 



Except for the t-test analysis of the ratio of^ questions t^o sentences 
per pag^, all^ thB results a*e observational rather than statistical. j 
The t-test results indicate, significant differences be t^en ajj^p air- 
vise contrasts of the four texts except for the Modern Biology versus 



the BSCS Blue Version in which the difference Vas not significant. 
With regdrd to the other variables examined, the following evaluative 

a 

differences were noted. Comparing the percentages of experiential 

-t . 
versus non-rexperlmental questions, the BSCS Green Version (BSCS-GV) 

has 71 perc^at experiential question; the Modern Biology (MB) 31 



perc^at exp< 
:; BSCS ' Yell 



percent; B^CS' Yellow Version (BSCS-YV) 52 percent; and 'the BSCS Blue 
Version (BSCS-BV) 58 .percent For the placement of question the MB 
text places the four categories abouf evenlyr*BSCS-GV has a high \ 
number of captional questions and few terminal questions. BSCS-YV 
ask? few terminal questions. All except the BSCS-GV place „ questions 5 
in context. Of the five types of experiential questions all texts 



used focusing questions extensively. MB asked the most rhetorical 
questions and the least valuing. BSCS-BV asked fewer direct inf or - 
mat ion questions than the other texts, but more, open-ended and 
indirect information questions. Regarding the , process- type ques- 
tions essentially no observing questions and few organizing and - 1 
experimenting questions were asked. However, ctie laboratory 
investigations were not iricluded in the study. Most of the BSCS-GV 
process questions involved contaunicating and applying while the 
BSCS-BV stressed inferring questions. ^ 
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Int erpr e t a t ions 

] - 

Significant differences between the ratio of question to sentences 
were found for all t-test pairwise comparisons except for Modern 
Biolftgy versus BSCS Blue Version , The. ratio of question to sentence 
was highest "for BSCS Green Version followed bjf BSCS Yellow .Version , - 
BSCS Blue Version and Modern Biology . 

The value of the particular questioning patterns of each textbook J 
is dependent upon the assumptions made about questioning. If examined 4 
from a Piagetian focus, which indicates that direct and active exper- 
ience is, valuable to learning, the BSCS-GV would be most desirable 
since it has the most experient iaily based questions. If asking a # 
high proportion of questions is important , then BSCS Greeri and Yellow 
are valuable. 1 

If the ^position of questions is a critical factor* beginning or end 

rather than contextual, then nqne of tlje texts is desirable^ Th£ _ 

effect of captibnal questions (B$CS-GV) has not been researched. All 

/ . v 

texts ask a rather low number of "higher level questions" arid *a^l are* 

relatively deficient in asking higher order inquiry-process questions.. . 

If balance of types of process questions is' important, then again all 

f - 
texts would rank low. - - 

The authojrs express a fieed for more research to clarify the relative 
values of certain types of questions. However, because questions 
are .considered to be stimuli to inquiry, it is felt that examining I 
in- text, questions is a legitimate, but neglected, basis for evaluat- 
ing textbooks. ^ * » 



&J£dZ$^ : ■ "... ^ - 

' • ' " ABSTRACTOR'S ANALYSIS .'• 



This study provides an int e res ting J and useful addition to the area of 
research related to questioning-. .Although it is not truly an experi- 
mental study, Lowery and Leonard examine^ question types and patterns 
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that have been dealt with experimentally in other studies. * In part-ic- 
ular, they look at in- text questions in regard to qu^pLon type and to 
position or plapementj^E questions rfithin the written material as well 
as the frequency of questions^to sentences per page. Research related 
to the former suggest that it Would be appropriate to examine the text- 
book an4 its relationship to the processes of inquiry. 

Conceptually what this stuciy might do is provide a ,f rameworlo f or iden- 
tifying research questions that would require further experimental 
examination. Wheh discussing the implications of the study, the 1 
authors defined the desirability of the f our texts^in accordance* 
with the assumptions of related questioning research.. For example, 
the placement of questions research indicates that question's placed 
iat the beginning or end of paragraphs have greater value to readers 
than those "in^context and, therefore, none of the texts is desirable 
since none place their questions at the beginning or the end of para- 
graphs. These various assumptions /day provide a rationale for future 
research issues, but extreme cautioti^ould be used if generalizations 
are. made that wpuld bridge the gap between in-text questions and. ques^ 
tloning research that, dekls with learning- outcomes. Essentially' the 
question of t^xtbo^k questions ^and effect ive learning outcomes has not 
yet been^askfe'd expeiftmentally. a ^ . 



The contribution of this study, is that it simply look?, at an area of 
questioning in frequent, use, i.e., the textbook, and provides some 
baseline data which is, in itself , necessary. It is time that someone 
.looked at questions, where and how they are as opposed to how they 
should be. ^ 

Because this was not actually an experimental study, in that it had 
no -manipulated* variables, it had no* Experimental design. The cate- 
gories for classifying questions were Established and the questions 
were subsequently assigned to the various categories and tabulated 
for the four textbooks. The only statistical test employed was a t- 

test for the pairvijse comparison of* the^ ratio of questions to 

>• • • • * * ■ 

sentences per p^ge, f or the four texts* Frequency tables were \ used 

•' ' ' • 
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to report ^ie results 'of types of questions asked (rhetorical, focus- 
ing, etc.) .and 'kind of process, of learning flues tieps (observing, 
organizing, itif erring, .*tc .) . ' # ^ ' 

' V 

Th* authors make a case for not subjecting the d if fetenc^S between 
^categories to statistical analysis because of an^ assumption that r a 
10 flprceht sampling is representative of the whole textboolc and, thus, 
the question of difference is an evaluative judgment rather than a 
statistical one^ There* is some room for disagreement with thjs idea. , 
Any time a sample is u^ed rather than thi whole population, there is 
the 'chance that it is not representative of the whole "population. The 
reason for using a statistical test to' begin with , is to give the 
researcher*an idea 6f ^iow big^that chance ma/ be/ The, Authors stated 
that the number of questions in. the textbooks were concentrated* in 
different sections of t?he books and that the placement of concentra- 
tion varie<Ji amoilg the four textbooks. It then nutynot be -valid to 
assume that the placement of fquestibn types is .necessarily unifprmly 
spaced throughout the texts. ^ pn the contrary, it would .appear that 
a systematic/ approach for inserting questions into the textbook was 
riot" used' af all, tjlus-making it necessary to lcjok at ,tke differences 
statistically^ as well as ev^luatively. . K 

One other question that, could be raised rega^in^ the' statistical analy 
sis concerns the^use of pairwise t-test rather than a one-way analysis 
of variance. : It would be interesting to know whether or not the 
differences found to be significant would hold up in cases where the 
t value was low* in a slightly more rigorous ted?. *' 

v • 9 * 

r ; 

The report was well vritten and was organized in & way to answer the 
ba^icjiuestiQns usually asked of a research paper. t The. authors did 
not attempt to generalize beyond the findings which were appropriate 
in light of the fact that most of the findings were not' subjected to 
statistical, analysis. The value of the^study -is that it seems to-be 
th6 first' to examine the role of the textbook and its relation to the 
processes^ of inquiry. Most studies of questioning have used specially 
prepared materials and did not offer suggestions for textbook. 



application. this study examines in- text questions 

asks how t tile 

research could be applied. ^ ^ ■« ^ >• 

Herein lies the suggestion for future research. The textbook questions 
could be experimentally manipulated fdr examining effectiveness on ^ 
learning outcomes. • It would be worthwhile to know how the textbook 
is used by the teacher to enhance, the learning process. If textbooks 
ire infrequently used,- it might not matter about the v types and number 
of questions incorporated. However, if the text is central to the 
instructional process, the""Tpjj^stion : of questioning becomes very ' 
Important. Vl would be nice fog: writers and researchers ^to get 
together to design* texts that would use questions appropriately and 
effectively.' -N^ .-' • v > ■ , 



